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SPECTRAL THEORY AND REPRESENTATIONS 
OF NILPOTENT GROUPS 



P. LeVY-BrUHL, a. MOHAMED, and J. NOURRIGAT 

Abstract. We give an estimate of the number N(X) of eigenvalues < A for the image 
under an irreducible representation of the "sublaplacian" on a stratified nilpotent Lie 
algebra. We also give an estimate for the trace of the heat-kernel associated with this 
operator. The estimates are formulated in term of geometrical objects related to the 
representation under consideration. An important particular case is the Schrodinger 
equation with polynomial electrical and magnetical fields. 



1. Introduction 

We first consider the Schrodinger operator with polynomial electrical and mag- 
netical fields 

n 



where Aj and V are real polynomials on R", of degree < r, and with V > 0. 
We define Bjk = dAj/dxk — dAk/dxj and assume that there is no rotation of the 
coordinates axis making all the Bjk and V independent of one of the Xi's. This 
case has been investigated in [9]: define 

Mix) = + 

|a|<r ot,j,k 

Mix,0^\^\+M{x). 

Let N{X) be the number of eigenvalues of P < A and iVo(A) the volume in M^" of 
the set of points {x, £,) such that M{x, < A. Then there is a C > 1, independent 
of A such that C-iiVo(C-iA) < A^(A) < CNo{CX) for all A > 0. A more precise 
equivalent in particular cases is given in [13]. This result was proved before, in a 
different formulation, by Fefferman, when the Aj^s are zero [1]. If V is the square 
of a polynomial, P is of the form 7r(— A) where A is a sublaplacian on a stratified 
r-step nilpotent Lie group and tt an irreducible representation [2, 9]. In the general 
case, {V > 0), our following results needs a very small change to be applied. This 
example is one of the motivations for the present generalization. 
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2. Statement of the results 

Let © be a stratified r-steps nilpotent Lie algebra. In other words, we assume 
that can be written as a direct sum of subspaces &j (1 < j < r) such that 

[(&j,ek]c&j+k if j + k<r, 
[<3,0k] = O ifi + fc>r, 

and such that 25 is generated as Lie algebra by the subspace ©i. We choose a basis 
Xi,...Xp of (Si, and we shall be interested in spectral properties of the image 
under irreducible representations of the sublaplacian A = — J2^=i -^j- 

Let TT be an irreducible, unitary, nontrivial representation of the connected simply 
connected group exp© associated to 0. It is well known that the operator 7r(— A), 
defined on the space 5^ of C°° vectors of the representation tt, has a unique selfad- 
joint extension, still denoted 7r(— A), whose spectrum is a sequence (Aj) {j G N) of 
positive eigenvalues, such that Xj < Xj+i and +oo. Let us denote by iV(A) 

the number of eigenvalues Xj < X. Our first goal is to give an estimate for N{X) in 
terms of geometrical objects associated to the representation tt. 

We denote by 5t {t > 0) the natural dilations of &; that is, the linear maps defined 
by 6tiX) = PX if X e C5j. Let also 6* be the dual of (S, S* the transpose of St, 
and III III a homogeneous norm on (5*, i.e., a nonnegative continuous, subadditive 
function on 0*, only vanishing at the origin and satisfying |||<5(Z||| = tp||| if t > 
and I e 0*. 

By the Kirillov theory, the representation tt is associated with an orbit 0„ of the 
coadjoint representation of the group G = exp0 in 0*. The G invariant measures 
on 0„ are proportional, and we denote by the canonical one; that is, with the 
correct normalization for the character formula [4, 12]. For each A > 0, we set 

No{X) = fi{l e O^, mf < A) . 

We can now state the estimate for N{X). 

Theorem 1. There exists a constant C > 1, independent of X, and of the repre- 
sentation IT such that 

C-^Nq{C-'^X) < N{X) < CNq{CX), for allX>0. 

Manchon gives in [7] the proof of a conjecture of Karasev-Maslov [3] . His result 

does not apply to our operator, but rather to —'^jY^, where (Yj) is a basis of 
as a vector space. Such an operator is also selfadjoint with compact resolvent, 
and A''(A) is given by a Weyl type formula. Such a formula does not make sense in 
general in our case, the integral involved in it being nonconvergent. 

It is also well known, since tt is irreducible, that for t > 0, the operator exp(— t7r(A)) 
is trace-class. We can estimate its trace 

Z{t) = Tr(exp(-<7r(A))) 

by the following result, using the function 

Zo{t)= [ exp(-t|||/f)dMO- 
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Theorem 2. There exists a constant C > 1, independent of t > 0, and of the 
representation w such that 

C-^Zo{Ct) < Z{t) < CZo{C-H) . 

As noticed in the introduetion. our results can be used for the Schrodinger op- 
erator with polynomial electrical and magnetical fields. 

3. Sketch of the proof 

Remark. We give here a sketch of the proof of Theorem 1. The reader may consult 
[6] for the details. 

We construct Hilbert spaces of sequences controlling the norm of the Sobolev 
spaces associated with 7r(— A) and use the minimax formula. 

For this construction, we need a suitable set of functions in L^(R"). The prop- 
erties of these functions have been suggested both by the ideas of Perelomov [11] 
(coherent states) and of Meyer [8] (wavelets). 

We realize the representation tt in the following form: for X G (&, 7r{X) is a 
differential operator on R", 

n{X) = Ai{X)d/dxi+ A2ixi,X)d/dx2 

(*) 

H h An{xi,. . . ,Xn-l,X)d/dXn + iB{x,X), 

where the Aj and B are real linear forms in X and polynomials in a; € K", Ai 
is independent of x, and Aj only depends on {xi, . . . ,Xj-i). For every finite se- 
quence / = {ii, . . . ,im) of positive integers smaller than p, 'it{Xi) is the iterated 
commutator 

TTiXi) = (ad7r(XiJ) • • • (ad7r(X,„_ J)7r(X,„) . 

Let n{X){x,(,) be the complete symbol of the operator n{X). The orbit O-j^ is the 
set of linear forms X —iTr{X){x, ^) for (x, ^) in R^", and the canonical measure 
/Lt is then dxd£^. We can also choose the homogeneous norm 

Ill/Ill =Eiu^i)r/|'|- 

The functions Mt^{x,^) and M^(a;) are defined by 

M^{x,0 = E k(X/)(a;,0|i/l^l and M^{x) = miM^{x,0- 

\I\<r ^ 

For the representation tt under consideration, M^(a;) > 0. We prove the theorems 

using this "concrete" realization of tt. Wc arc not able to construct an orthonormal 
basis of so that the Sobolev spaces associated to our operator are l"^ weighted 
spaces, but the following result is a substitute and allows the use of minimax formu- 
las. Let p{x, ^) = x, {x, £_) G M". For every finite sequence (ai, . . . , am) of positive 
integers smaller than p, we define 7r(X") = Tr{Xai) ■ • ■TT{Xa,„ ). and for 

«eCo~(R"): |lu||.„,.= I E M^'^M' 

y |a|=m 
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Theorem 3. "Coherent states.''^ Let n be an induced representation of & realized 
in the form (*) with M^{x) > for all a; e M". {tt is not necessarily irreducible.) 
For every a > small enough, there exists [ipj) and {^j) {j e N) in C^(M"), a 
sequence fij of com,pact sets in M^", a point {xj,£,j) G flj, and C > with: 

1. u = ^jiV-j for all u m ^^(M"). 

2. (supp^j) C (supp)V'j C pi^j) for all j e N. 

3. Every finite subset A of N contains a part B such that #B > {#A)/C and 
such that for every sequence {Xj) of complex numbers 



4. For every integer m > 0, there exists Cm > with the properties : If u G 
Cq°(M") is of the form u = Xjtpj {Xj S C not necessarily given by {u,^j)) 
then IImIIto^ < C,nJ2j l^jl'^^^'i^jT^j)'^"^ ■ V i''^ addition Xj = {u,^j) then we have 

\{u,^-)\'M^ix,,^,r- < CM\L.- 

5. ffi.^" C Uj(%) '^^'^ every (x,^) is contained in at mostC of the ri^'s. Further, 
1/C< Vol(Oj) < C. 

6. For{x,C) inilj, we have the estimate {l/C)Mj,{xj,S^j) < M7r(x,^) < CM„{xj,^j). 

7. C and the constants C„i only depend, on a and © not on tt. 

The construction of the coherent states is first performed locahy: we associate 
to each a; € M" a representation ax, equivalent to tt such that ax has the form 
(*) with Mcr^(O) ~ M,r{x) and the coefficients of the polynomials in the expression 
of ax{St^{X)) where t = Mt^{x) are bounded, independently of x and tt. Using 
Fourier series and induction on the dimension n, "local coherent states" for a^ are 
constructed. Certain properties in Theorem 3 arc only satisfied if u is supported in 
the ball of radius a, centered at 0. The coherent states are then obtained by means 
of a partition of unity, modeled on the symplectic diffeomorphisms related to the 
intertwining operators between tt and a^- 

In the proof, the following theorem is used (the notation is as before). 

Proposition 4. There exists C > such that for every representation tt of the 
form {*) and all u e S{W) 

E ||M.(xr-H7r(X>f <C||«||^,,. 

|a|<m 

In this form, this theorem requires 6 to be stratified, however, the proof of 
Lemma 4.5 in [10] works without this hypothesis. The function M is then more 
complicated, reflecting the structure of ©. It is possible to formulate our present 
results in this set-up. The same method also applies to more general operators 
than the model presented here; we need only a control of {'k{P)u, u) in terms of the 
II 11^,^ norm. 
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